System and methods for the automatic transmission of new, high affinity media

ABSTRACT

A system and methods for the automatic transmission of new, high affinity media to a user are provided. In connection with a system that convergently merges perceptual and digital signal processing analysis of media entities for purposes of classifying the media entities, various means are provided to a user for automatically extracting media entities that represent a high (or low) affinity state/space for the user in connection with the generation of a high affinity playlist, channel or station. Techniques for providing a dynamic recommendation engine and techniques for rating media entities are also included are also included. Once a high affinity state/space is identified, the high affinity state/space may be persisted for the user from experience to experience.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.09/900,230, filed Jul. 6, 2001, which claims the benefit of U.S.Provisional Application Ser. No. 60/216,106, filed Jul. 6, 2000, thecontents of all are incorporated herein by reference in its entirety.

FIELD OF THE INVENTION

The present invention relates to a system and methods for the automatictransmission of new, high affinity media to users of computing devicesconnected to a network.

BACKGROUND OF THE INVENTION

Classifying information that has subjectively perceived attributes orcharacteristics is difficult. When the information is one or moremusical compositions, classification is complicated by the widelyvarying subjective perceptions of the musical compositions by differentlisteners. One listener may perceive a particular musical composition as“hauntingly beautiful” whereas another may perceive the same compositionas “annoyingly twangy.”

In the classical music context, musicologists have developed names forvarious attributes of musical compositions. Terms such as adagio,fortissimo, or allegro broadly describe the strength with whichinstruments in an orchestra should be played to properly render amusical composition from sheet music. In the popular music context,there is less agreement upon proper terminology. Composers indicate howto render their musical compositions with annotations such as brightly,softly, etc., but there is no consistent, concise, agreed-upon systemfor such annotations.

As a result of rapid movement of musical recordings from sheet music topre-recorded analog media to digital storage and retrieval technologies,this problem has become acute. In particular, as large libraries ofdigital musical recordings have become available through global computernetworks, a need has developed to classify individual musicalcompositions in a quantitative manner based on highly subjectivefeatures, in order to facilitate rapid search and retrieval of largecollections of compositions.

Musical compositions and other information are now widely available forsampling and purchase over global computer networks through onlinemerchants such as AMAZON.COM®, BARNESANDNOBLE.COM®, CDNOW.COM®, etc. Aprospective consumer can use a computer system equipped with a standardWeb browser to contact an online merchant, browse an online catalog ofpre-recorded music, select a song or collection of songs (“album”), andpurchase the song or album for shipment direct to the consumer. In thiscontext, online merchants and others desire to assist the consumer inmaking a purchase selection and desire to suggest possible selectionsfor purchase. However, current classification systems and search andretrieval systems are inadequate for these tasks.

A variety of inadequate classification and search approaches are nowused. In one approach, a consumer selects a musical composition forlistening or for purchase based on past positive experience with thesame artist or with similar music. This approach has a significantdisadvantage in that it involves guessing because the consumer has nofamiliarity with the musical composition that is selected.

In another approach, a merchant classifies musical compositions intobroad categories or genres. The disadvantage of this approach is thattypically the genres are too broad. For example, a wide variety ofqualitatively different albums and songs may be classified in the genreof “Popular Music” or “Rock and Roll.”

In still another approach, an online merchant presents a search page toa client associated with the consumer. The merchant receives selectioncriteria from the client for use in searching the merchant's catalog ordatabase of available music. Normally the selection criteria are limitedto song name, album title, or artist name. The merchant searches thedatabase based on the selection criteria and returns a list of matchingresults to the client. The client selects one item in the list andreceives further, detailed information about that item. The merchantalso creates and returns one or more critics' reviews, customer reviews,or past purchase information associated with the item.

For example, the merchant may present a review by a music critic of amagazine that critiques the album selected by the client. The merchantmay also present informal reviews of the album that have been previouslyentered into the system by other consumers. Further, the merchant maypresent suggestions of related music based on prior purchases of others.For example, in the approach of AMAZON.COM®, when a client requestsdetailed information about a particular album or song, the systemdisplays information stating, “People who bought this album also bought. . . ” followed by a list of other albums or songs. The list of otheralbums or songs is derived from actual purchase experience of thesystem. This is called “collaborative filtering.”

However, this approach has a significant disadvantage, namely that thesuggested albums or songs are based on extrinsic similarity as indicatedby purchase decisions of others, rather than based upon objectivesimilarity of intrinsic attributes of a requested album or song and thesuggested albums or songs. A decision by another consumer to purchasetwo albums at the same time does not indicate that the two albums areobjectively similar or even that the consumer liked both. For example,the consumer might have bought one for the consumer and the second for athird party having greatly differing subjective taste than the consumer.As a result, some pundits have termed the prior approach as the “greaterfools” approach because it relies on the judgment of others.

Another disadvantage of collaborative filtering is that output data isnormally available only for complete albums and not for individualsongs. Thus, a first album that the consumer likes may be broadlysimilar to second album, but the second album may contain individualsongs that are strikingly dissimilar from the first album, and theconsumer has no way to detect or act on such dissimilarity.

Still another disadvantage of collaborative filtering is that itrequires a large mass of historical data in order to provide usefulsearch results. The search results indicating what others bought areonly useful after a large number of transactions, so that meaningfulpatterns and meaningful similarity emerge. Moreover, early transactionstend to over-influence later buyers, and popular titles tend toself-perpetuate.

In a related approach, the merchant may present information describing asong or an album that is prepared and distributed by the recordingartist, a record label, or other entities that are commerciallyassociated with the recording. A disadvantage of this information isthat it may be biased, it may deliberately mischaracterize the recordingin the hope of increasing its sales, and it is normally based oninconsistent terms and meanings.

In still another approach, digital signal processing (DSP) analysis isused to try to match characteristics from song to song, but DSP analysisalone has proven to be insufficient for classification purposes. WhileDSP analysis may be effective for some groups or classes of songs, it isineffective for others, and there has so far been no technique fordetermining what makes the technique effective for some music and notothers. Specifically, such acoustical analysis as has been implementedthus far suffers defects because 1) the effectiveness of the analysis isbeing questioned regarding the accuracy of the results, thus diminishingthe perceived quality by the user and 2) recommendations can only bemade if the user manually types in a desired artist or song title fromthat specific website. Accordingly, DSP analysis, by itself, isunreliable and thus insufficient for widespread commercial or other use.

Accordingly, there is a need for an improved method of classifyinginformation that is characterized by the convergence of subjective orperceptual analysis and DSP acoustical analysis criteria. With such aclassification technique, it would be further desirable to leveragesong-by-song analysis and matching capabilities to automatically and/ordynamically personalize a high affinity network-based experience for auser. In this regard, there is a need for a mechanism that can enable aclient to automatically retrieve information about one or more musicalcompositions, user preferences, ratings, or other sources of mappings topersonalize an experience for listener(s).

SUMMARY OF THE INVENTION

In view of the foregoing, the present invention provides a system andmethods for the automatic transmission of new, high affinity mediatailored to a user. In connection with a system that convergently mergesperceptual and digital signal processing analysis of media entities forpurposes of classifying the media entities, the present inventionprovides various means to a user for automatically extracting mediaentities that represent a high (or low) affinity state/space for theuser in connection with the generation of a high affinity playlist,channel or station. Techniques for providing a dynamic recommendationengine and techniques for rating media entities are also included. Oncea high affinity state/space is identified, the high affinity state/spacemay be persisted for a user from experience to experience.

Other features of the present invention are described below.

BRIEF DESCRIPTION OF THE DRAWINGS

The system and methods for the automatic transmission of new, highaffinity media are further described with reference to the accompanyingdrawings in which:

FIG. 1 is a block diagram representing an exemplary network environmentin which the present invention may be implemented;

FIG. 2 is a high level block diagram representing the media contentclassification system utilized to classify media, such as music, inaccordance with the present invention;

FIG. 3 is block diagram illustrating an exemplary method of thegeneration of general media classification rules from analyzing theconvergence of classification in part based upon subjective and in partbased upon digital signal processing techniques;

FIG. 4 illustrates an embodiment of the present invention whereby amedia station, is tailored to a user through the user's specification ofa piece of media;

FIG. 5 illustrates an embodiment of the present invention whereby amedia station is tailored to a user through the user's specification ofpartial specifiers;

FIG. 6 illustrates an embodiment of the present invention whereby amedia station is tailored to a user through multi-level musicorganization and a one step process for providing a personalized highaffinity station;

FIG. 7 illustrates an embodiment of the present invention whereby amedia station is tailored to a user through a one-step personalized “GetMore” station reprogram technique;

FIG. 8 illustrates an exemplary implementation of a one-steppersonalized station replay, or one-step personalization based upon aprevious media entity selection;

FIG. 9 illustrates an exemplary process of the operation of adynamically updated recommendation engine in accordance with the presentinvention;

FIG. 10 illustrates an exemplary process wherein a user's preferenceprofile is dynamically updated; and

FIG. 11 illustrates an exemplary ratings-based process in accordancewith the present invention for dynamically updating a recommendationengine.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Overview

The present invention provides a system and method whereby new, highaffinity media are transmitted to a user of a networked computingdevice. The present invention leverages the song-by-song analysis andmatching capabilities of modem music matching and classificationtechniques. For example, commonly assigned U.S. Pat. No. 6545,209, filedJul. 5, 2001, hereinafter the analysis and matching system, describesnovel techniques for analyzing and matching based upon musical propertymappings, such as may be defined for a song or a media station. Theanalysis and matching system enables searching of an analysis andmatching database, based upon high affinity input mappings extracted orcaptured in accordance with the present invention, for the purpose ofreturning songs that are correlated to the input mappings. The presentinvention takes such technique(s) yet another step further byautomatically personalizing a high affinity network-based mediaexperience, such as a Web-based radio experience of a computing device,for a user. In this regard, the present invention provides an array ofdynamically-generated or one-step personalization functionalityadvancements that support the automatic transmission of new, highaffinity media to an end user of any network-enabled computing devicevia wired or wireless means.

Exemplary Computer and Network Environments

One of ordinary skill in the art can appreciate that a computer 110 orother client device can be deployed as part of a computer network. Inthis regard, the present invention pertains to any computer systemhaving any number of memory or storage units, and any number ofapplications and processes occurring across any number of storage unitsor volumes. The present invention may apply to an environment withserver computers and client computers deployed in a network environment,having remote or local storage. The present invention may also apply toa standalone computing device, having access to appropriateclassification data.

FIG. 1 illustrates an exemplary network environment, with a server incommunication with client computers via a network, in which the presentinvention may be employed. As shown, a number of servers 10 a, 10 b,etc., are interconnected via a communications network 14, which may be aLAN, WAN, intranet, the Internet, etc., with a number of client orremote computing devices 110 a, 110 b, 110 c, 110 d, 110 e, etc., suchas a portable computer, handheld computer, thin client, networkedappliance, or other device, such as a VCR, TV, and the like inaccordance with the present invention. It is thus contemplated that thepresent invention may apply to any computing device in connection withwhich it is desirable to provide classification services for differenttypes of content such as music, video, other audio, etc. In a networkenvironment in which the communications network 14 is the Internet, forexample, the servers 10 can be Web servers with which the clients 110 a,110 b, 110 c, 110 d, 110 e, etc. communicate via any of a number ofknown protocols such as hypertext transfer protocol (HTTP).Communications may be wired or wireless, where appropriate. Clientdevices 110 may or may not communicate via communications network 14,and may have independent communications associated therewith. Forexample, in the case of a TV or VCR, there may or may not be a networkedaspect to the control thereof. Each client computer 110 and servercomputer 10 may be equipped with various application program modules 135and with connections or access to various types of storage elements orobjects, across which files may be stored or to which portion(s) offiles may be downloaded or migrated. Any server 10 a, 10 b, etc. may beresponsible for the maintenance and updating of a database 20 inaccordance with the present invention, such as a database 20 for storingclassification information, music and/or software incident thereto.Thus, the present invention can be utilized in a computer networkenvironment having client computers 110 a, 110 b, etc. for accessing andinteracting with a computer network 14 and server computers 10 a, 10 b,etc. for interacting with client computers 110 a, 110 b, etc. and otherdevices 111 and databases 20.

Classification

In accordance with one aspect of the present invention, a uniqueclassification is implemented which combines human and machineclassification techniques in a convergent manner, from which a canonicalset of rules for classifying music may be developed, and from which adatabase, or other storage element, may be filled with classified songs.With such techniques and rules, radio stations, studios and/or anyoneelse with an interest in classifying music can classify new music. Withsuch a database, music association may be implemented in real time, sothat playlists or lists of related (or unrelated if the case requires)media entities may be generated. Playlists may be generated, forexample, from a single song and/or a user preference profile inaccordance with an appropriate analysis and matching algorithm performedon the data store of the database. Nearest neighbor and/or othermatching algorithms may be utilized to locate songs that are similar tothe single song and/or are suited to the user profile.

FIG. 2 illustrates an exemplary classification technique in accordancewith the present invention. Media entities, such as songs 210, fromwherever retrieved or found, are classified according to humanclassification techniques at 220 and also classified according toautomated computerized DSP classification techniques at 230. 220 and 230may be performed in either order, as shown by the dashed lines, becauseit is the marriage or convergence of the two analyses that provides astable set of classified songs at 240. As discussed above, once such adatabase of songs is classified according to both human and automatedtechniques, the database becomes a powerful tool for generating songswith a playlist generator 250. A playlist generator 250 may takeinput(s) regarding song attributes or qualities, which may be a song oruser preferences, and may output a playlist, recommend other songs to auser, filter new music, etc. depending upon the goal of using therelational information provided by the invention. In the case of a songas an input, techniques for human-based classification, automatedcomputerized DSP classification, or some combination thereof asdescribed above, are utilized to determine the attributes, qualities,likelihood of success, etc. of the song. In the case of user preferencesas an input, a search may be performed for songs that match the userpreferences to create a playlist or make recommendations for new music.In the case of filtering new music, the rules used to classify the songsin database 240 may be leveraged to determine the attributes, qualities,genre, likelihood of success, etc. of the new music. In effect, therules can be used as a filter to supplement any other decision makingprocesses with respect to the new music.

FIG. 3 illustrates an embodiment of the invention, which generatesgeneralized rules for a classification system. A first goal is to traina database with enough songs so that the human and automatedclassification processes converge, from which a consistent set ofclassification rules may be adopted, and adjusted to accuracy. First, at305, a general set of classifications are agreed upon in order toproceed consistently i.e., a consistent set of terminology is used toclassify music in accordance with the present invention. At 310, a firstlevel of expert classification is implemented, whereby experts classifya set of training songs in database 300. This first level of expert isfewer in number than a second level of expert, termed herein a groover,and in theory has greater expertise in classifying music than the secondlevel of expert or groover. The songs in database 300 may originate fromanywhere, and are intended to represent a broad cross-section of music.At 320, the groovers implement a second level of expert classification.There is a training process in accordance with the invention by whichgroovers learn to consistently classify music, for example to 92-95%reproducibility of attribute classification across different groovers.The groover scrutiny reevaluates the classification of 310, andreclassifies the music at 325 if the groover determines thatreassignment should be performed before storing the song in humanclassified training song database 330.

Before, after or at the same time as the human classification process,the songs from database 300 are classified according to digital signalprocessing (DSP) techniques at 340. Exemplary classifications for songsinclude, inter alia, tempo, sonic, melodic movement and musicalconsonance characterizations. Classifications for other types of media,such as images, video or software are also contemplated, as they wouldfollow an analogous process of classification, although the specificattributes measured would obviously be different. The quantitativemachine classifications and qualitative human classifications for agiven piece of media, such as a song, are then placed into what isreferred to herein as a classification chain, which may be an array orother list of vectors, wherein each vector contains the machine andhuman classification attributes assigned to the piece of media. Machinelearning classification module 350 marries the classifications made byhumans and the classifications made by machines, and in particular,creates a rule when a trend meets certain criteria. For example, ifsongs with heavy activity in the frequency spectrum at 3 kHz, asdetermined by the DSP processing, are also characterized as ‘jazzy’ byhumans, a rule can be created to this effect. The rule would be, forexample: songs with heavy activity at 3 kHz are jazzy. Thus, when enoughdata yields a rule, machine learning classification module 350 outputs arule to rule set 360. While this example alone may be anoversimplification, since music patterns are considerably more complex,it can be appreciated that certain DSP analyses correlate well to humananalyses.

However, once a rule is created, it is not considered a generalizedrule. The rule is then tested against like pieces of media, such assong(s), in the database 370. If the rule works for the generalizationsong(s) 370, the rule is considered generalized. The rule is thensubjected to groover scrutiny 380 to determine if it is an accurate ruleat 385. If the rule is inaccurate according to groover scrutiny, therule is adjusted. If the rule is considered to be accurate, then therule is kept as a relational rule e.g., that may classify new media.

The above-described technique thus maps a pre-defined parameter space toa psychoacoustic perceptual space defined by musical experts. Thismapping enables content-based searching of media, which in part enablesthe automatic transmission of high affinity media content, as describedbelow.

Automatic Transmission of High Affinity Media Content

The present invention relates generally to the broadcasting or renderingof media from a network-enabled computing device, such as a radio, or aradio broadcast rendered via a network portal, such as a Web site. Thepersonalization process works via an interplay of features with theabove-described song analysis and matching system. A user makes aspecific choice that represents a high affinity state/space for theuser, such as a choice representing something desirable to the specificuser about a piece or set of media. The choice may be the choice of apiece of media itself, a choice regarding a characteristic of a song orsongs more generally, or a choice regarding a characteristic of theuser. The specific choice within any of the features can be representedas a mapping along a set of fundamental musical properties that capturesa user's psychoacoustic preferences. The song analysis and matchingsystem then scans the database for other musical entities that have asimilar mapping of musical properties. These newly found entities arethen automatically returned to the user. The return of these resultsleverages the user's original choice to provide the user with anexperience that tailors itself automatically to the user's specificpsychoacoustic preferences, and hence prolongs the user's high affinitystate/space. The linking works because every piece of audio mediatransmitted to the user is mapped on a set of fundamental musicalproperties that in sum can represent a user's high affinity,state/space.

Existing artist and genre-based ways to specify a radio stream are verybroad and hence have not captured a user's specific psychoacousticpreferences, and hence cannot as effectively prolong a user's highaffinity stat/space.

In connection with the above-described song analysis, classification andmatching processes, the present invention provides advancements in thearea of automatic personalization of a user's media experience, all ofwhich allow the user to get a highly targeted set of music via only asmall amount of effort. By leveraging the song analysis and matchingtechniques, users can accurately “ask” for music for which there will behigh affinity. A user specifies psychoacoustic preferences with theinformation he or she presents to the song analysis and matching system.This “asking” process takes a variety of forms and is described in moredetail in commonly assigned U.S. Patent Appln. No. [Attorney Docket No.:MSFT-0585] with respect to how user's specific preference(s) aretranslated into an actual playlist.

FIGS. 4 through 7 illustrate different embodiments in which a userspecifies psychoacoustic preferences, which then form the basis for asearch of the matching and analysis database, which in turn results inthe automatic transmission of high affinity media to the user.

FIG. 4 illustrates an embodiment of the present invention whereby amedia station, such as a radio station, is tailored to a user throughthe user's specification of a piece of media, such as a song. From thecharacteristics of the song, a high affinity playlist is generated. At400, a user finds a computing device having a user interface inaccordance with the present invention for accessing any of a variety oftypes of media, such as music. The user interface does not have tofollow any particular format, and a user may use any known input devicefor entering data into the system. At 410, a user searches for, locates,finds or otherwise designates via an input device a familiar song thatthe user finds pleasing psychoacoustically. At 420, the selection of themedia link itself begins the automatic personalization process, althoughan affirmative action on the part of the user could also be implementedto begin the process. At 430, as a result of the start of the automaticpersonalization process, an immediate search of media analysis andmatching database for similarly matched media is performed. At 440, theresults of step 430, namely the return of media similarly matched to thesong selected, are built into the present or actual playlist of themedia station. At 450, the user experiences other media with similarproperties as the piece of familiar media content via the playlistformed at 440. At 460, the user can opt to prolong the high affinitystate/space associated with the selected piece of familiar mediacontent.

Thus, the user may launch or instantiate a radio station on anetwork-enabled computing device in a one-step personalization process,whereby the process automatically plays a set of songs with similarfundamental musical properties as the chosen song. This process connectssongs for which a user has high affinity to the base song by findingother songs that have similar mappings and hence a song likelihood ofcontinuing the user's high affinity state/space. Automatically returnedis the related playlist of songs. The success of the above process, inpart, hinges on the classification scheme utilized at the front end ofthe present invention, wherein both perceptual analysis techniques andacoustic analysis techniques are utilized, providing a degree ofmatching success in connection with the media analysis and matchingdatabase.

FIG. 5 illustrates an embodiment of the present invention whereby amedia station, such as a radio station, is tailored to a user throughthe user's specification of partial-song intuitive psychoacousticspecifiers. A user can specify the type of music that the user wants tohear by defining only a partial element of a song. In other words, auser may ask for music targeted on a subset of fundamental musicalproperties. In one implementation, at 500, a user finds a computingdevice having a user interface in accordance with the present inventionfor accessing any of a variety of types of media, such as music. At 510,the user specifies base setting(s) or media qualities, independent ofname or artist, that represent a current high affinity state/space forthe user. Thus, the user specifies intuitive, as opposed to solely byartist or genre, music descriptors that the user already understands,such as mood descriptors (happy, sad, energetic, groovy, soothing),tempo descriptors (fastest, fast, moderate, slow, slowest), or weightdescriptors (heaviest, heavy, moderate, light, lightest), orcombinations of the aforementioned descriptors and/or other likedescriptors. Alternatively, these music descriptors may be combined withfurther restricting criteria, such as music by a particular artist orwithin a particular genre only. An exemplary restriction includes arestriction to the “fastest, happy songs by the artist Bob Dylan.” Whenfinished specifying, the user may send the descriptor set to thedatabase for matching via a one step personalization process. Thedescriptor set is directly mapped into the database via the analysis andmatching system, and songs with similar psychoacoustic properties as thespecified descriptors are automatically returned at 520 and 530 forexperience by the user at 540, although the returned songs have norestrictions for any non-set or non-specified properties. In thismanner, the user can have a playlist generated via a limited musicalproperty mapping without thinking according to a larger unit ofanalysis—song, album, artist, genre. At 550, the user may choose toprolong the high affinity state/space associated with the selected pieceof familiar media content.

FIG. 6 illustrates an embodiment of the present invention whereby amedia station, such as a radio station, is tailored to a user throughmulti-level music organization and a one step process for providing apersonalized high affinity station. A user can specify the type of musicthe user wants to hear via high-affinity matching with various levels ofmusic classification, including but not limited to: partial-song, song,album, artist, genre. These various levels exist below, at, and abovethe song level of classification. In one implementation, at 600, a userfinds a computing device having a user interface in accordance with thepresent invention for accessing any of a variety of types of media, suchas music. At 610, the user specifies base setting(s) or media qualities,which may include song name, album, artist, genre, etc., as well as theintuitive descriptors described previously, that represent a currenthigh affinity state/space for the user. At 620, the user may add thesebase setting(s) to a media ‘channel’ built at the network location. At630, the base setting(s) are processed, organized and stored accordingto the cross-level entities represented thereby. Thus, a user may grouphigh affinity preferences across multiple levels of music classificationinto personal “stations”. Additionally, at 640, for furtherspecification and in recognition that not all preferences are equal, auser may specify an inclination towards or a frequency for entities toemphasize the relative importance of the preference to the user. In anexemplary embodiment, the frequency with which each station entry hasmatching songs returned is based upon the weighting preference given bythe user, for example, “A lot”, “Some”, “A little”, or “Never.” Sincethe mappings for all preferences entered are captured via the personalstation, selecting the personal station begins one step personalizationof media to the user at 650. Selecting the personal station causes themappings for the entered preferences to automatically run through theanalysis and matching system at 660, and returned at 670 is a highaffinity mixed playlist with songs that are psychoacoustically similarto entries on the station. U.S. Patent Appln. No. [Attorney Docket No.:MSFT-0585] describes more specific methods for playlist constructionbased upon frequency of preferences and the like. At 680, the userexperiences other media with similar properties as the preferences ofthe base setting(s) via the playlist formed at 670. At 690, the user canopt to prolong the high affinity state/space associated with the newlyformed channel generated from the base setting(s).

FIG. 7 illustrates an embodiment of the present invention whereby amedia station, such as a radio station, is tailored to a user through aone-step personalized “Get More” station reprogram technique. Forexample, this technique could supplement the personal station listeningexperience described in connection with FIG. 6, or operate upon anyplaylist currently broadcast to a user based upon an underlying set ofpreferences, however specified. While listening to a particular song, auser can specify that the current playing song better corresponds to ahigher affinity space/state at that time than the current stationsetting. The user requests to “get more” which instructs the system tofind more songs like the current one. In effect, and partly inrecognition of a user that doesn't know exactly how to specify what heor she likes to a tee, but knows what he or she likes when the userhears it, the present embodiment allows a user to specify the propertiesof the current playing song, by selecting the current playing song, inorder to hone the user's preferences. This then captures the musicalproperty mapping of the currently playing song, automatically runs themapping through the analysis and matching system, and returns a highaffinity playlist of songs that replaces the existing station. Thisprocess automatically connects the user's high affinity towards thecurrent song with other songs that have similar mappings and hence astrong likelihood of continuing the user's high affinity state/space.

Thus, in exemplary detail, at 700, a user finds a computing devicehaving a user interface in accordance with the present invention foraccessing any of a variety of types of media, such as music. At 705, theuser searches and finds a media entity, such as a song, of high affinityto the user. At 710, the link associated with the song starts theautomatic personalization process as described in connection with FIG.4, although any of the above-described embodiments may be used to buildan initial high affinity playlist. At 715, a search of the analysis andmatching database is performed to retrieve similarly mapped songs forbuilding into an initial playlist at 720. At 725, the user beginslistening to the newly retrieved media from the initial playlist. At730, the user decides that the current song playing corresponds to ahigher affinity state/space than what the playlist offers moregenerally. Thus, at 735, the user selects a link or other inputcomponent to indicate that the user would like to specify his or herpreferences more in line with the presently playing song, which beginsan automatic personalization process that further hones a playlist tothe user's newly specified preference for the presently playing song. At740, the mappings of the currently playing song are run through theanalysis and matching database, to return new media entities for a newplaylist at 745. At 750, the user experiences other media with similarpsychoacoustic properties as the “Get More” selected song. At 755, theuser can opt to prolong the high affinity state/space associated withthe newly formed playlist generated as a consequence of the “Get More”selected song. In an alternate embodiment, the mappings represented bythe “Get More” song may be used to supplement the mappings representedby the initial playlist, such that the “tweaking” of the playlist ismore subtle than generating a brand new playlist.

The “Get More” mapping may be easily extended to refer to the intuitivemusic descriptors, such as mood, tempo, and weight, to provide specifictailoring of future playlists along one of those dimensions. Forexample, one of ordinary skill in the art can readily appreciate animplementation of “Get Faster” and “Get Slower” controls, the activationof which may indicate a user's affinity for music whose correspondingattribute (tempo) lies more in the specified direction. As with the “GetMore” control, the resulting personalization may apply to either thecreation of a new playlist, or a further honing of the currently-playingone.

Supplementing the above techniques, the present invention may store auser's historical record of stations, pieces of media selected and/orother user preferences. Thus, the methods of the present inventioninclude tracking a user's historical record of station settings and thesongs played in those stations. Thus, in the case of a radio stationimplemented by a network enabled computing device, the present inventionstores a historical playlist record of all songs played in all stationsever listened to at the radio station by a specific user. This record isstored even when the user has left the site and then returns e.g., thismay occur via cookie(s) if user is not logged in, and by login name whena user does log in. This historical record allows for several automaticpersonalization improvements that further leverage the capabilities ofthe song analysis and matching system.

FIG. 8 illustrates an exemplary implementation of a one-steppersonalized station replay, or one-step personalization based upon aprevious media entity selection. By accessing the historical record, auser can track decisions that the user has made. Then, with a one-steppersonalization process, the user can restart, select or link to any oldmedia station or media entity in the record. With this input, themusical mapping properties of the station or media entity arere-captured and automatically run through the analysis and matchingsystem. Returned is a playlist of media entities that immediatelyreplaces the existing station or playlist; however, the new set of songsin the already played station is substantially different from theoriginal set of songs for that setting, as described in U.S. PatentAppln. No. [Attorney Docket No. MSFT-0585], but the new songs equallymatch the psychoacoustic properties of the station setting. Thus, thepresent invention provides the ability to leverage the song analysis andmatching system to get equally personalized psychoacoustic songs, butnot the same songs as before.

Thus, in exemplary detail, at 800, a user finds a computing devicehaving a user interface in accordance with the present invention foraccessing any of a variety of types of media, such as music. At 805, theuser variously makes decisions as to songs and stations for which theuser has a high affinity. At 810, the station and/or song choices aretracked to form a historical record. At 815, a user may view thehistorical record generated at 810 in connection with the user's choicesof 805. At 820, a user decides that a certain station or song in thehistorical is desirable. At 825, the link associated with the song orstation selected at 820 starts an automatic personalization process thatforms a playlist according to the selected station or song mappings. At830, a search of the analysis and matching database is performed toretrieve similarly mapped songs for building into a playlist. At 835,new media entities are returned for a new playlist. As mentioned, thenew set of songs in the already played station is different from theoriginal set of songs for that setting. At 840, the user experiencesother media with similar psychoacoustic properties as the selected songor station. At 845, the user can opt to prolong the high affinitystate/space associated with the newly formed playlist.

FIG. 9 illustrates an exemplary process of the operation of adynamically updated recommendation engine in accordance with the presentinvention. By analyzing a user's historical record and leveraging thisinformation with the song analysis and matching system, a userautomatically receives song recommendations that match trends seen inthe fundamental musical properties of the historical user record. Once arecord has begun, there is a dynamically updated analysis of the record.With every new station setting, the engine re-analyzes up to thetotality of the user's station decisions to extract core patterns orpsychoacoustic preferences seen in the record. This mapping has thepotential for dynamic morphing with every new station choice. Theanalysis and matching system then searches the database for otherentities with similar mappings. Automatically returned are highlytargeted songs that have similar psychoacoustic properties as the coremapping patterns. A user receives, by choice, all songs that fit themapping, those songs that both fit the mapping and have not been playedon the current playlist, or those songs that both fit the mapping andhave not been played on the radio in any playlist to date.

Thus, in exemplary detail, at 900, a user finds a computing devicehaving a user interface in accordance with the present invention foraccessing any of a variety of types of media, such as music. At 905, theuser makes a decision to play a first station for which the user has ahigh affinity. At 910, the station (and/or song choices) is tracked toform a historical record. At 915, a mapping of the selected firststation is captured and stored. At 920, the user makes a decision toplay a second station for which the user has a high affinity. At 925, amapping of the selected second station is captured and stored. At 930,mappings across the station settings are cross analyzed for the mostprominent psychoacoustic features that in aggregation representlongitudinal high affinity state/space for the user. This, for example,is accomplished through an analysis at 960 that records the mean andstandard deviation for each numerical fundamental used in theclassification chain. At 935, any new stations (third, fourth, etc.)added over time also have their mappings dynamically added to theanalysis and new prominent similarities morph from existing ones. At940, the up to date prominent similarities' mapping is run through theanalysis and matching database. At 945, the engine of the inventionautomatically returns media entities that are most similar to the up todate dynamic mapping sent to the database at 940. At 950, in anexemplary implementation, the user chooses to see all recommended mediaentities, those media entities not in the current playlist, or thosemedia entities never before seen at the network location or Site. At955, the presentation of recommended media entities initiates orprolongs the user's high affinity state/space associated with the newlychosen recommended playlist.

The present invention also may utilize a system of rating media entitiesthat leverages the analysis and matching system to personalize a user'sexperience. By linking to the analysis and matching database, thisrating system has capabilities beyond rating systems that compare oneuser's preferences to another's i.e., collaborative filtering systems.For example, in the context of music, these rating capabilities couldwork on a variety of rating scales, both active and passive, includingbut not limited to “hot/not” ratings, an “N-star rating scale” wherebythe number of stars selected is proportional to the user's affinity forthe music, implicit low affinity for skipped songs, most common soundslike query songs, most commonly played clips on radio/site, etc.Furthermore, users may specify ratings at higher levels of the datahierarchy, including but not limited to the album, artist, or genrelevel. These ratings would “bubble down” to the song contained therein;a rating of an artist, for example, would necessarily affect in aproportional manner the ratings of that artist's albums, which in turnwould proportionally affect the respective ratings of the songs on thosealbums.

FIG. 10 illustrates an exemplary process wherein a user's preferenceprofile is dynamically updated. By monitoring a user's ongoing ratings,the initial conditions for that user's high affinity state/space can berepresented. For example, when a song is rated positively, its mappingis recorded. By taking the aggregated mappings of positively ratedsongs, the analysis system may look for core psychoacoustic propertiesin the aggregation that define the initial conditions for that user'shigh affinity state/space. When a song is rated negatively, its mappingmay also be recorded. By taking the aggregated mappings of negativelyrated songs, the analysis system may look for core psychoacousticproperties in the aggregation that define the initial conditions for aparticular user's low-affinity state/space. With every rating, theseprofiles for high and low-affinity state/spaces are remapped on-the-fly.

This overall high/low-affinity preference profile may then be utilizedas a basis for dynamically seeding a playlist generator with likelihoodweightings for all potential songs in the roll-up. Songs matching thehigh affinity state are weighted as more likely to play. Songs matchingthe low-affinity state are weighted as less likely to play, or thesesongs are blocked from playing altogether if enough other songs exist togenerate a playlist of acceptable length. As mentioned, U.S. PatentAppln. No. [Attorney Docket No.: MSFT-0585] describes more specificmethods for playlist construction based upon frequency or weights ofpreferences, and the like.

Thus, in an exemplary implementation, at 1000, a user finds a computingdevice having a user interface in accordance with the present inventionfor accessing any of a variety of types of media, such as music. At1005, the user makes a decision to rate a media entity, such as a song,as representative of the user's high or low affinity state/space. Thismay be done at 1045 via an active rating or a passive rating. Activeratings are ratings that include action on the part of the user for thepurpose of assigning a rating, such as the user rating the song as goodor bad, hot or not, etc., the user assigning a rating from 1 to 10 tothe song, the user skipping a song thereby suggesting that the song isof low affinity for the user, and the like. Passive ratings may beextracted from actions on the part of the user, but these includeactions that are not done for the explicit purpose of assigning arating. Passive ratings, for example, might include identifying the mostfrequent “sounds like” queries made by the user, identifying the mostcommonly played songs by the user, identifying the most commonly skippedsongs by the user, and the like. At 1010, if the media entity rated at1005 is representative of the user's high affinity space, then a mappingof positively-rated media entity is captured and compared to existinghigh/low affinity mappings in the historical record. At 1015, if themedia entity rated at 1005 is representative of the user's low affinityspace, then a mapping of negatively-rated media entity is captured andcompared to existing low affinity mappings. At 1020, the engineautomatically builds or updates a preference profile corresponding tothe user's preferences for high affinity and low affinity psychoacousticproperties. At 1025, the user makes a decision to start a high affinitystation. At 1030, this causes a search of the analysis and matchingdatabase to be performed to find media entities that are similar todynamically updated preference profile built at 1020. At 1035, aplaylist is dynamically generated based upon seed mappings withlikelihood weightings for all potential media entities in the roll-up,wherein a high affinity profile corresponds to an increase in likelihoodand a low affinity profile corresponds to a decrease in likelihood. At1040, the user may opt to prolong the high affinity state/spaceassociated with the aggregated set of ratings over the user's entirehistory.

FIG. 11 illustrates another exemplary ratings-based process inaccordance with the present invention for dynamically updating arecommendation engine. By monitoring a user's ratings, that user'scurrent high affinity state/space is captured. When a song is rated, itsspecific psychoacoustic properties are mapped. If the rating ispositive, then the mapping is compared with the dynamically updatingrecommendation engine based on the user's historical record. If a corepsychoacoustic property exists in the positively rated song that is notrepresented in the dynamically-updated mapping, then the recommendationengine uses the analysis and matching system to search the database foradditional songs that have a similar mapping as this newly identifiedhigh affinity psychoacoustic property. Automatically returned arespecific, highly targeted songs that have similar psychoacousticproperties as the core mapping pattern. A user receives, by choice, allsongs that fit the mapping, those songs that both fit the mapping andhave not been played on the current playlist, or those songs that bothfit the mapping and have not been played on the radio in any playlist todate.

In an exemplary implementation, at 1100, a user finds a computing devicehaving a user interface in accordance with the present invention foraccessing any of a variety of types of media, such as music. At 1105,the user makes a decision to rate a media entity, such as a song, asrepresentative of the user's high or low affinity state/space. This maybe done at 1135 via an active rating, including but limited to suchexamples as the user rating the song as good or bad, hot or not, etc.,the user assigning a rating from 1 to 10 to the song, the user skippinga song thereby suggesting that the song is of low affinity for the user,and the like. At 1110, if the media entity rated at 1105 isrepresentative of the user's high affinity space, then a mapping ofpositively-rated media entity is captured and compared to existing highaffinity mappings in the historical record. At 1115, the prominentdistinctions in psychoacoustic properties between the rated media entityand the historical record are extracted. At 1120, the engineautomatically returns media entities that are most similar to thedynamic mapping of distinct psychoacoustic features updated at 1115. At1125, the user chooses to see all recommended media entities, thosesongs not in the current playlist or those songs never before seen atthe site. At 1130, the site presents the newly recommended entities thatcorrespond to the user's choice at 1125 may prolong the high affinitystate/space associated with the entities.

As mentioned above, the media contemplated by the present invention inall of its various embodiments is not limited to music or songs, butrather the invention applies to any media to which a classificationtechnique may be applied that merges perceptual (human) analysis withdigital signal processing (DSP) analysis for increased accuracy inclassification and matching.

The various techniques described herein may be implemented with hardwareor software or, where appropriate, with a combination of both. Thus, themethods and apparatus of the present invention, or certain aspects orportions thereof, may take the form of program code (i.e., instructions)embodied in tangible media, such as floppy diskettes, CD-ROMs, harddrives, or any other machine-readable storage medium, wherein, when theprogram code is loaded into and executed by a machine, such as acomputer, the machine becomes an apparatus for practicing the invention.In the case of program code execution on programmable computers, thecomputer will generally include a processor, a storage medium readableby the processor (including volatile and non-volatile memory and/orstorage elements), at least one input device, and at least one outputdevice. One or more programs are preferably implemented in a high levelprocedural or object oriented programming language to communicate with acomputer system. However, the program(s) can be implemented in assemblyor machine language, if desired. In any case, the language may be acompiled or interpreted language, and combined with hardwareimplementations.

The methods and apparatus of the present invention may also be embodiedin the form of program code that is transmitted over some transmissionmedium, such as over electrical wiring or cabling, through fiber optics,or via any other form of transmission, wherein, when the program code isreceived and loaded into and executed by a machine, such as an EPROM, agate array, a programmable logic device (PLD), a client computer, avideo recorder or the like, the machine becomes an apparatus forpracticing the invention. When implemented on a general-purposeprocessor, the program code combines with the processor to provide aunique apparatus that operates to perform the indexing functionality ofthe present invention. For example, the storage techniques used inconnection with the present invention may invariably be a combination ofhardware and software.

While the present invention has been described in connection with thepreferred embodiments of the various figures, it is to be understoodthat other similar embodiments may be used or modifications andadditions may be made to the described embodiment for performing thesame function of the present invention without deviating therefrom. Forexample, while exemplary embodiments of the invention are described inthe context of music data, one skilled in the art will recognize thatthe present invention is not limited to the music, and that the methodsof tailoring media to a user, as described in the present applicationmay apply to any computing device or environment, such as a gamingconsole, handheld computer, portable computer, etc., whether wired orwireless, and may be applied to any number of such computing devicesconnected via a communications network, and interacting across thenetwork. Furthermore, it should be emphasized that a variety of computerplatforms, including handheld device operating systems and otherapplication specific operating systems are contemplated, especially asthe number of wireless networked devices continues to proliferate.Therefore, the present invention should not be limited to any singleembodiment, but rather construed in breadth and scope in accordance withthe appended claims.

1. A method for matching a user's musical preferences, comprising:specifying at least one song to a content provider as representative ofthe user's musical preferences, wherein each song has an associatedaudio stream for rendering the song; analyzing, by a song analysis andmatching system, at least one portion of an audio stream associated witha song of the specified at least one song and mapping said at least oneportion to a set of fundamental musical properties; scanning a databaseusing the song analysis and matching system to find other songs thathave a similar mapping of fundamental musical properties; andautomatically playing at least one of said other songs to the user.
 2. Aone step song personalization process in accordance with the method ofclaim 1 in which the user requests a particular song which the userfinds pleasing and the song analysis and matching system automaticallyplays a set of songs with similar fundamental musical properties as thechosen song.
 3. A limited musical property song personalization processin accordance with the method of claim 1 in which the user furtherspecifies the user's musical preferences by defining a partial elementof a song wherein the element is selected from a group of song elementsincluding mood descriptors, tempo descriptors and weight descriptors andwherein said scanning includes finding other songs that further satisfythe conditions defined by said partial element.
 4. A multi-level musicalproperty song personalization process in accordance with the method ofclaim 1 in which the user further specifies the user's musicalpreferences by specifying a plurality of analysis elements selected froma group of analysis elements including at least one of a partial elementof a song, an album, an artist and a genre.
 5. A one step “get more”personalization process in accordance with the method of claim 1 inwhich the user, while listening to a particular song, transmits a “getmore” command resulting in the musical properties of the currentlyplaying song being captured by the analysis and matching system andautomatically playing to the user other songs that have a similarmapping of musical properties as the currently playing song.
 6. A onestep “get faster” personalization process in accordance with the methodof claim 1 in which for a given musical property, the user may indicatean affinity for music whose corresponding attribute lies more in aspecified direction for the musical property.
 7. A dynamically updatedrecommendation process in accordance with the method of claim 1 in whichthe user automatically receives recommendations that match at least onetrend detected by the song analysis and matching system.
 8. Adynamically updated user profile in accordance with the method of claim1 in which the song analysis and matching system determines a userprofile based on the historical record of past decisions made by theuser.
 9. A ratings-based dynamically updated recommendation process inaccordance with the method of claim 8 in which the song analysis andmatching system determines at least one trend in song selection anddynamically changes the user's profile based on the at least one trend,thereby making recommendations to the user.
 10. A method for matching auser's musical preferences, comprising: means for specifying at leastone song to a content provider as representative of the user's musicalpreferences, wherein each song has an associated audio stream forrendering the song; means for analyzing at least one portion of an audiostream associated with a song of the specified at least one song andmeans for mapping said at least one portion to a set of fundamentalmusical properties; means for scanning a database to find other songsthat have a similar mapping of fundamental musical properties; and meansfor automatically playing at least one of said other songs to the user.